Search AI Products and News

Researchers at the Hebrew University of Jerusalem recently discovered that in Retrieval Augmented Generation (RAG) systems, the number of documents processed significantly impacts language model performance, even when the total text length remains constant. The research team conducted experiments using 2,417 questions from the MuSiQue validation dataset, each linked to 20 Wikipedia paragraphs. Two to four paragraphs contained relevant answer information, with the remaining paragraphs serving as distractors. To study the impact of the number of documents, the team created multiple data partitions, progressively reducing the number of documents from 20 to...

Alibaba Tongyi Lab has open-sourced ViDoRAG, a novel visual document retrieval and question answering system. ViDoRAG boasts impressive accuracy, achieving a remarkable 79.4% in benchmark tests. This system represents a significant advancement in the field of computer vision and information retrieval.

The RAG system tutorial has been released! This system is based on arXiv papers as a context source and provides links to the source papers used in generating answers. Tutorial link: https://colab.research.google.com/drive/1Lc8eq8P87JjzUhbYb33_c7h7njsWb-hn#scrollTo=eCSBhP4FxOg3 . The tutorial thoroughly demonstrates the process of implementing the RAG system, including obtaining paper text, preprocessing and chunking PDF documents using Unstructured, creating a ChromaDB retriever, setting up RAG